Brightness illusions drive a neuronal response in the primary visual cortex under top-down modulation.

Brightness illusions are a powerful tool in studying vision, yet their neural correlates are poorly understood. Based on a human paradigm, we presented illusory drifting gratings to mice. Primary visual cortex (V1) neurons responded to illusory gratings, matching their direction selectivity for real gratings, and they tracked the spatial phase offset between illusory and real gratings. Illusion responses were delayed compared to real gratings, in line with the theory that processing illusions requires feedback from higher visual areas (HVAs). We provide support for this theory by showing a reduced V1 response to illusions, but not real gratings, following HVAs optogenetic inhibition. Finally, we used the pupil response (PR) as an indirect perceptual report and showed that the mouse PR matches the human PR to perceived luminance changes. Our findings resolve debates over whether V1 neurons are involved in processing illusions and highlight the involvement of feedback from HVAs.

Parietal theta burst TMS does not modulate bistable perception.

The role of the parietal cortex in perceptual awareness and in resolving perceptual ambiguity is unsettled. Early influential transcranial magnetic stimulation studies have revealed differences in conscious perception following parietal stimulation, fuelling the notion that parietal cortex causally contributes to resolving perceptual ambiguity. However, central to this conclusion is the reliability of the method employed. Several prior studies have revealed opposing effects, such as shortening, lengthening, or no effect on multistable perceptual transitions following parietal stimulation. Here we addressed the reliability of continuous theta-burst stimulation (cTBS) on parietal cortex on the perception of bistable stimuli. We conducted three cTBS experiments that were matched to prior experiments in terms of stimuli, stimulation protocol, and target site, and used a higher number of participants. None of our cTBS experiments replicated prior cTBS results. The only experiment using individual functional localizers led to weak effects, while the two others led to null results. Individual variability of motor cortex cTBS did not predict parietal cTBS effects. In view of recent reports of highly variable cTBS effects over motor cortex, our results suggest that cTBS is particularly unreliable in modulating bistable perception when applied over parietal cortex.

Visual cortex: Big data analysis uncovers food specificity.

In visual cortex, anatomically distinct patches respond to distinct categories, such as faces or text. New research confirms this parcellation using unsupervised analysis of functional magnetic resonance imaging data obtained from humans viewing tens of thousands of images, discovering one more preference: for food.

The neural coding of face and body orientation in occipitotemporal cortex.

Face and body orientation convey important information for us to understand other people's actions, intentions and social interactions. It has been shown that several occipitotemporal areas respond differently to faces or bodies of different orientations. However, whether face and body orientation are processed by partially overlapping or completely separate brain networks remains unclear, as the neural coding of face and body orientation is often investigated separately. Here, we recorded participants' brain activity using fMRI while they viewed faces and bodies shown from three different orientations, while attending to either orientation or identity information. Using multivoxel pattern analysis we investigated which brain regions process face and body orientation respectively, and which regions encode both face and body orientation in a stimulus-independent manner. We found that patterns of neural responses evoked by different stimulus orientations in the occipital face area, extrastriate body area, lateral occipital complex and right early visual cortex could generalise across faces and bodies, suggesting a stimulus-independent encoding of person orientation in occipitotemporal cortex. This finding was consistent across functionally defined regions of interest and a whole-brain searchlight approach. The fusiform face area responded to face but not body orientation, suggesting that orientation responses in this area are face-specific. Moreover, neural responses to orientation were remarkably consistent regardless of whether participants attended to the orientation of faces and bodies or not. Together, these results demonstrate that face and body orientation are processed in a partially overlapping brain network, with a stimulus-independent neural code for face and body orientation in occipitotemporal cortex.

Disrupting Short-Term Memory Maintenance in Premotor Cortex Affects Serial Dependence in Visuomotor Integration.

Human behavior is biased by past experience. For example, when intercepting a moving target, the speed of previous targets will bias responses in future trials. Neural mechanisms underlying this so-called serial dependence are still under debate. Here, we tested the hypothesis that the previous trial leaves a neural trace in brain regions associated with encoding task-relevant information in visual and/or motor regions. We reasoned that injecting noise by means of transcranial magnetic stimulation (TMS) over premotor and visual areas would degrade such memory traces and hence reduce serial dependence. To test this hypothesis, we applied bursts of TMS pulses to right visual motion processing region hV5/MT+ and to left dorsal premotor cortex (PMd) during intertrial intervals of a coincident timing task performed by twenty healthy human participants (15 female). Without TMS, participants presented a bias toward the speed of the previous trial when intercepting moving targets. TMS over PMd decreased serial dependence in comparison to the control Vertex stimulation, whereas TMS applied over hV5/MT+ did not. In addition, TMS seems to have specifically affected the memory trace that leads to serial dependence, as we found no evidence that participants' behavior worsened after applying TMS. These results provide causal evidence that an implicit short-term memory mechanism in premotor cortex keeps information from one trial to the next, and that this information is blended with current trial information so that it biases behavior in a visuomotor integration task with moving objects.SIGNIFICANCE STATEMENT Human perception and action are biased by the recent past. The origin of such serial bias is still not fully understood, but a few components seem to be fundamental for its emergence: the brain needs to keep previous trial information in short-term memory and blend it with incoming information. Here, we present evidence that a premotor area has a potential role in storing previous trial information in short-term memory in a visuomotor task and that this information is responsible for biasing ongoing behavior. These results corroborate the perspective that areas associated with processing information of a stimulus or task also participate in maintaining that information in short-term memory even when this information is no longer relevant for current behavior.

Consciousness: What is the role of prefrontal cortex?

Consciousness, its neural underpinnings, and the role of frontal cortex are highly debated topics. New evidence shows that human frontal cortex can bias conscious perception. What does this really mean about its contribution to consciousness?

Separated and overlapping neural coding of face and body identity.

Recognising a person's identity often relies on face and body information, and is tolerant to changes in low-level visual input (e.g., viewpoint changes). Previous studies have suggested that face identity is disentangled from low-level visual input in the anterior face-responsive regions. It remains unclear which regions disentangle body identity from variations in viewpoint, and whether face and body identity are encoded separately or combined into a coherent person identity representation. We trained participants to recognise three identities, and then recorded their brain activity using fMRI while they viewed face and body images of these three identities from different viewpoints. Participants' task was to respond to either the stimulus identity or viewpoint. We found consistent decoding of body identity across viewpoint in the fusiform body area, right anterior temporal cortex, middle frontal gyrus and right insula. This finding demonstrates a similar function of fusiform and anterior temporal cortex for bodies as has previously been shown for faces, suggesting these regions may play a general role in extracting high-level identity information. Moreover, we could decode identity across fMRI activity evoked by faces and bodies in the early visual cortex, right inferior occipital cortex, right parahippocampal cortex and right superior parietal cortex, revealing a distributed network that encodes person identity abstractly. Lastly, identity decoding was consistently better when participants attended to identity, indicating that attention to identity enhances its neural representation. These results offer new insights into how the brain develops an abstract neural coding of person identity, shared by faces and bodies.

Magic, Bayes and wows: A Bayesian account of magic tricks.

Magic tricks have enjoyed an increasing interest by scientists. However, most research in magic focused on isolated aspects of it and a conceptual understanding of magic, encompassing its distinct components and varieties, is missing. Here, we present an account of magic within the theory of Bayesian predictive coding. We present the "wow" effect of magic as an increase in surprise evoked by the prediction error between expected and observed data. We take into account prior knowledge of the observer, attention, and (mis-)direction of perception and beliefs by the magician to bias the observer's predictions and present a simple example for the modelling of the evoked surprise. The role of misdirection is described as everything that aims to maximize the surprise a trick evokes by the generation of novel beliefs, the exploitation of background knowledge and attentional control of the incoming information. Understanding magic within Bayesian predictive coding allows unifying all aspects of magic tricks within one framework, making it tractable, comparable and unifiable with other models in psychology and neuroscience.

Investigating holistic face processing within and outside of face-responsive brain regions.

It has been shown that human faces are processed holistically (i.e. as indecomposable wholes, rather than by their component parts) and this holistic face processing is linked to brain activity in face-responsive brain regions. Although several brain regions outside of the face-responsive network are also sensitive to relational processing and perceptual grouping, whether these non-face-responsive regions contribute to holistic processing remains unclear. Here, we investigated holistic face processing in the composite face paradigm both within and outside of face-responsive brain regions. We recorded participants' brain activity using fMRI while they performed a composite face task. Behavioural results indicate that participants tend to judge the same top face halves as different when they are aligned with different bottom face halves but not when they are misaligned, demonstrating a composite face effect. Neuroimaging results revealed significant differences in responses to aligned and misaligned faces in the lateral occipital complex (LOC), and trends in the anterior part of the fusiform face area (FFA2) and transverse occipital sulcus (TOS), suggesting that these regions are sensitive to holistic versus part-based face processing. Furthermore, the retrosplenial cortex (RSC) and the parahippocampal place area (PPA) showed a pattern of neural activity consistent with a holistic representation of face identity, which also correlated with the strength of the behavioural composite face effect. These results suggest that neural activity in brain regions both within and outside of the face-responsive network contributes to the composite-face effect.

Eye-selective fMRI activity in human primary visual cortex: Comparison between 3 ​T and 9.4 ​T, and effects across cortical depth.

The primary visual cortex of humans contains patches of neurons responding preferentially to stimulation of one eye (the ocular dominance columns). Multiple previous studies attempted to detect their activity using fMRI. The majority of these fMRI studies used magnetic field strengths of 4 â€‹T and higher. However, there have been reports of reliable eye-selective activations at 3 â€‹T as well. In this study we investigated the possibility of detecting eye-selective V1 activity using high-resolution GE-EPI fMRI at 3 â€‹T and sub-millimeter resolution fMRI at ultrahigh 9.4 â€‹T magnetic field strengths with acquisition parameters optimized for each field strength. High-resolution fMRI at 9.4 â€‹T also allowed us to examine the eye-selectivity responses across the cortical depth, which are expected to be strongest in the middle layers. We observed a substantial increase in the percentage of eye-selective voxels, as well as a doubling in run-to-run consistency of eye preference at ultrahigh field compared to 3 â€‹T. We also found that across cortical depth, eye selectivity increased towards the superficial layers, and that signal contrast increased while noise remained nearly constant towards the surface. The depth-resolved results are consistent with a distortion of spatial specificity of the GE-EPI signal by ascending venules and large draining veins on the cortical surface. The effects of larger vessels cause increasing signal amplitude, but also displacement of the maximum BOLD signal relative to neural activity. In summary, our results show that increase in spatial resolution, reduced partial volume effects, and improved sensitivity at 9.4 â€‹T allow for better detection of eye-selective signals related to ocular dominance columns. However, although ultrahigh field yields higher sensitivity to the ocular dominance signal, GE-EPI still suffers from specificity issues, with a prominent signal contribution at shallow depths from larger cortical vessels.

Conscious perception of flickering stimuli in binocular rivalry and continuous flash suppression is not affected by tACS-induced SSR modulation.

The content of conscious perception is known to correlate with steady-state responses (SSRs), yet their causal relationship remains unclear. Can we manipulate conscious perception by directly interfering with SSRs through transcranial alternating current stimulation (tACS)? Here, we directly addressed this question in three experiments involving binocular rivalry and continuous flash suppression (CFS). Specifically, while participants (N = 24) viewed either binocular rivalry or tried to detect stimuli masked by CFS, we applied sham or real tACS across parieto-occipital cortex at either the same or a different frequency and phase as an SSR eliciting flicker stimulus. We found that tACS did not differentially affect conscious perception in the forms of predominance, CFS detection accuracy, reaction time, or metacognitive sensitivity, confirmed by Bayesian statistics. We conclude that tACS application at frequencies of stimulus-induced SSRs does not have perceptual effects and that SSRs may be epiphenomenal to conscious perception.

Decoding subcategories of human bodies from both body- and face-responsive cortical regions.

Our visual system can easily categorize objects (e.g. faces vs. bodies) and further differentiate them into subcategories (e.g. male vs. female). This ability is particularly important for objects of social significance, such as human faces and bodies. While many studies have demonstrated category selectivity to faces and bodies in the brain, how subcategories of faces and bodies are represented remains unclear. Here, we investigated how the brain encodes two prominent subcategories shared by both faces and bodies, sex and weight, and whether neural responses to these subcategories rely on low-level visual, high-level visual or semantic similarity. We recorded brain activity with fMRI while participants viewed faces and bodies that varied in sex, weight, and image size. The results showed that the sex of bodies can be decoded from both body- and face-responsive brain areas, with the former exhibiting more consistent size-invariant decoding than the latter. Body weight could also be decoded in face-responsive areas and in distributed body-responsive areas, and this decoding was also invariant to image size. The weight of faces could be decoded from the fusiform body area (FBA), and weight could be decoded across face and body stimuli in the extrastriate body area (EBA) and a distributed body-responsive area. The sex of well-controlled faces (e.g. excluding hairstyles) could not be decoded from face- or body-responsive regions. These results demonstrate that both face- and body-responsive brain regions encode information that can distinguish the sex and weight of bodies. Moreover, the neural patterns corresponding to sex and weight were invariant to image size and could sometimes generalize across face and body stimuli, suggesting that such subcategorical information is encoded with a high-level visual or semantic code.

Human V6 Integrates Visual and Extra-Retinal Cues during Head-Induced Gaze Shifts.

A key question in vision research concerns how the brain compensates for self-induced eye and head movements to form the world-centered, spatiotopic representations we perceive. Although human V3A and V6 integrate eye movements with vision, it is unclear which areas integrate head motion signals with visual retinotopic representations, as fMRI typically prevents head movement executions. Here we examined whether human early visual cortex V3A and V6 integrate these signals. A previously introduced paradigm allowed participant head movement during trials, but stabilized the head during data acquisition utilizing the delay between blood-oxygen-level-dependent (BOLD) and neural signals. Visual stimuli simulated either a stable environment or one with arbitrary head-coupled visual motion. Importantly, both conditions were matched in retinal and head motion. Contrasts revealed differential responses in human V6. Given the lack of vestibular responses in primate V6, these results suggest multi-modal integration of visual with neck efference copy signals or proprioception in V6.

A Generic Mechanism for Perceptual Organization in the Parietal Cortex.

Our visual system's ability to group visual elements into meaningful entities and to separate them from others is referred to as scene segmentation. Visual motion often provides a powerful cue for this process as parallax or coherence can inform the visual system about scene or object structure. Here we tested the hypothesis that scene segmentation by motion cues relies on a common neural substrate in the parietal cortex. We used fMRI and a set of three entirely distinct motion stimuli to examine scene segmentation in the human brain. The stimuli covered a wide range of high-level processes, including perceptual grouping, transparent motion, and depth perception. All stimuli were perceptually bistable such that percepts alternated every few seconds while the physical stimulation remained constant. The perceptual states were asymmetric, in that one reflected the default (nonsegmented) interpretation, and the other the non-default (segmented) interpretation. We confirmed behaviorally that upon stimulus presentation, the default percept was always perceived first, before perceptual alternations ensued. Imaging results showed that across all stimulus classes perceptual scene-segmentation was associated with an increase of activity in the posterior parietal cortex together with a decrease of neural signal in the early visual cortex. This pattern of activation is compatible with predictive coding models of visual perception, and suggests that parietal cortex hosts a generic mechanism for scene segmentation.SIGNIFICANCE STATEMENT Making sense of cluttered visual scenes is crucial for everyday perception. An important cue to scene segmentation is visual motion: slight movements of scene elements give away which elements belong to the foreground or background or to the same object. We used three distinct stimuli that engage visual scene segmentation mechanisms based on motion. They involved perceptual grouping, transparent motion, and depth perception. Brain activity associated with all three mechanisms converged in the same parietal region with concurrent deactivation of early visual areas. The results suggest that posterior parietal cortex is a hub involved in structuring visual scenes based on different motion cues, and that feedback modulates early cortical processing in accord with predictive coding theory.

Real-motion signals in human early visual cortex.

Eye movements induce visual motion that can complicate the stable perception of the world. The visual system compensates for such self-induced visual motion by integrating visual input with efference copies of eye movement commands. This mechanism is central as it does not only support perceptual stability but also mediates reliable perception of world-centered objective motion. In humans, it remains elusive whether visual motion responses in early retinotopic cortex are driven by objective motion or by retinal motion associated with it. To address this question, we used fMRI to examine functional responses of sixteen visual areas to combinations of planar objective motion and pursuit eye movements. Observers were exposed to objective motion that was faster, matched or slower relative to pursuit, allowing us to compare conditions that differed in objective motion velocity while retinal motion and eye movement signals were matched. Our results show that not only higher level motion regions such as V3A and V6, but also early visual areas signaled the velocity of objective motion, hence the product of integrating retinal with non-retinal signals. These results shed new light on mechanisms that mediate perceptual stability and real-motion perception, and show that extra-retinal signals related to pursuit eye movements influence processing in human early visual cortex.

Human V4 Activity Patterns Predict Behavioral Performance in Imagery of Object Color.

Color is special among basic visual features in that it can form a defining part of objects that are engrained in our memory. Whereas most neuroimaging research on human color vision has focused on responses related to external stimulation, the present study investigated how sensory-driven color vision is linked to subjective color perception induced by object imagery. We recorded fMRI activity in male and female volunteers during viewing of abstract color stimuli that were red, green, or yellow in half of the runs. In the other half we asked them to produce mental images of colored, meaningful objects (such as tomato, grapes, banana) corresponding to the same three color categories. Although physically presented color could be decoded from all retinotopically mapped visual areas, only hV4 allowed predicting colors of imagined objects when classifiers were trained on responses to physical colors. Importantly, only neural signal in hV4 was predictive of behavioral performance in the color judgment task on a trial-by-trial basis. The commonality between neural representations of sensory-driven and imagined object color and the behavioral link to neural representations in hV4 identifies area hV4 as a perceptual hub linking externally triggered color vision with color in self-generated object imagery.SIGNIFICANCE STATEMENT Humans experience color not only when visually exploring the outside world, but also in the absence of visual input, for example when remembering, dreaming, and during imagery. It is not known where neural codes for sensory-driven and internally generated hue converge. In the current study we evoked matching subjective color percepts, one driven by physically presented color stimuli, the other by internally generated color imagery. This allowed us to identify area hV4 as the only site where neural codes of corresponding subjective color perception converged regardless of its origin. Color codes in hV4 also predicted behavioral performance in an imagery task, suggesting it forms a perceptual hub for color perception.

Integration of visual and non-visual self-motion cues during voluntary head movements in the human brain.

Our phenomenological experience of the stable world is maintained by continuous integration of visual self-motion with extra-retinal signals. However, due to conventional constraints of fMRI acquisition in humans, neural responses to visuo-vestibular integration have only been studied using artificial stimuli, in the absence of voluntary head-motion. We here circumvented these limitations and let participants to move their heads during scanning. The slow dynamics of the BOLD signal allowed us to acquire neural signal related to head motion after the observer's head was stabilized by inflatable aircushions. Visual stimuli were presented on head-fixed display goggles and updated in real time as a function of head-motion that was tracked using an external camera. Two conditions simulated forward translation of the participant. During physical head rotation, the congruent condition simulated a stable world, whereas the incongruent condition added arbitrary lateral motion. Importantly, both conditions were precisely matched in visual properties and head-rotation. By comparing congruent with incongruent conditions we found evidence consistent with the multi-modal integration of visual cues with head motion into a coherent "stable world" percept in the parietal operculum and in an anterior part of parieto-insular cortex (aPIC). In the visual motion network, human regions MST, a dorsal part of VIP, the cingulate sulcus visual area (CSv) and a region in precuneus (Pc) showed differential responses to the same contrast. The results demonstrate for the first time neural multimodal interactions between precisely matched congruent versus incongruent visual and non-visual cues during physical head-movement in the human brain. The methodological approach opens the path to a new class of fMRI studies with unprecedented temporal and spatial control over visuo-vestibular stimulation.

Binocular rivalry transitions predict inattention symptom severity in adult ADHD.

Attention deficit and hyperactivity disorder (ADHD) is a prevalent childhood disorder that is often maintained throughout the development and persists into adulthood. Established etiology models suggest that deficient inhibition underlies the core ADHD symptoms. While experimental evidence for impaired motor inhibition is overwhelming, little is known about the sensory inhibition processes, their changes throughout the development, and the relationship to ADHD symptoms. Here, we used the well-established binocular rivalry (BR) paradigm to investigate for the very first time the inhibitory processes related to visual perception in adults with ADHD. In BR, perception alternates between two dichoptically presented images throughout the viewing period, with shorter dominant percept durations and longer transition periods indicating poorer suppression/inhibition. Healthy controls (N = 28) and patients with ADHD (N = 32) were presented with two dissimilar images (orthogonal gratings) separately to each eye through a mirror stereoscope and asked to report their perceptual experiences. There were no differences between groups in any of the BR markers. However, an association between transition durations and symptom severity emerged in the ADHD group. Importantly, an exploratory multiple regression analysis revealed that inattention symptoms were the sole predictor for the duration of transition periods. The lack of impairments to sensory inhibition in adult, but not pediatric ADHD may reflect compensatory changes associated with development, while a correlation between inhibition and inattention symptoms may reveal an invariant core of the disorder.

Invariance of surface color representations across illuminant changes in the human cortex.

A central problem in color vision is that the light reaching the eye from a given surface can vary dramatically depending on the illumination. Despite this, our color percept, the brain's estimate of surface reflectance, remains remarkably stable. This phenomenon is called color constancy. Here we investigated which human brain regions represent surface color in a way that is invariant with respect to illuminant changes. We used physically realistic rendering methods to display natural yet abstract 3D scenes that were displayed under three distinct illuminants. The scenes embedded, in different conditions, surfaces that differed in their surface color (i.e. in their reflectance property). We used multivariate fMRI pattern analysis to probe neural coding of surface reflectance and illuminant, respectively. While all visual regions encoded surface color when viewed under the same illuminant, we found that only in V1 and V4α surface color representations were invariant to illumination changes. Along the visual hierarchy there was a gradient from V1 to V4α to increasingly encode surface color rather than illumination. Finally, effects of a stimulus manipulation on individual behavioral color constancy indices correlated with neural encoding of the illuminant in hV4. This provides neural evidence for the Equivalent Illuminant Model. Our results provide a principled characterization of color constancy mechanisms across the visual hierarchy, and demonstrate complementary contributions in early and late processing stages.

Connectivity Reveals Sources of Predictive Coding Signals in Early Visual Cortex During Processing of Visual Optic Flow.

Superimposed on the visual feed-forward pathway, feedback connections convey higher level information to cortical areas lower in the hierarchy. A prominent framework for these connections is the theory of predictive coding where high-level areas send stimulus interpretations to lower level areas that compare them with sensory input. Along these lines, a growing body of neuroimaging studies shows that predictable stimuli lead to reduced blood oxygen level-dependent (BOLD) responses compared with matched nonpredictable counterparts, especially in early visual cortex (EVC) including areas V1-V3. The sources of these modulatory feedback signals are largely unknown. Here, we re-examined the robust finding of relative BOLD suppression in EVC evident during processing of coherent compared with random motion. Using functional connectivity analysis, we show an optic flow-dependent increase of functional connectivity between BOLD suppressed EVC and a network of visual motion areas including MST, V3A, V6, the cingulate sulcus visual area (CSv), and precuneus (Pc). Connectivity decreased between EVC and 2 areas known to encode heading direction: entorhinal cortex (EC) and retrosplenial cortex (RSC). Our results provide first evidence that BOLD suppression in EVC for predictable stimuli is indeed mediated by specific high-level areas, in accord with the theory of predictive coding.